Gender Fairness within the Force Concept Inventory

Research on the test structure of the Force Concept Inventory (FCI) has largely ignored gender, and research on FCI gender effects (often reported as "gender gaps") has seldom interrogated the structure of the test. These rarely-crossed streams of research leave open the possibility that the FCI may not be structurally valid across genders, particularly since many reported results come from calculus-based courses where 75% or more of the students are men. We examine the FCI considering both psychometrics and gender disaggregation (while acknowledging this as a binary simplification), and find several problematic questions whose removal decreases the apparent gender gap. We analyze three samples (total $N_{pre}=5,391$, $N_{post}=5,769$) looking for gender asymmetries using Classical Test Theory, Item Response Theory, and Differential Item Functioning. The combination of these methods highlights six items that appear substantially unfair to women and two items biased in favor of women. No single physical concept or prior experience unifies these questions, but they are broadly consistent with problematic items identified in previous research. Removing all significantly gender-unfair items halves the gender gap in the main sample in this study. We recommend that instructors using the FCI report the reduced-instrument score as well as the 30-item score, and that credit or other benefits to students not be assigned using the biased items.Comment: 18 pages, 3 figures, 5 tables; submitted to Phys. Rev. PE

Examining the effects of testwiseness in conceptual physics evaluations

Testwiseness is defined as the set of cognitive strategies used by a student that is intended to improve his or her score on a test regardless of the test’s subject matter. Questions with elements that may be affected by testwiseness are common in physics assessments, even in those which have been extensively validated and widely used as evaluation tools in physics education research. The potential effect of several elements of testwiseness were analyzed for questions in the Force Concept Inventory (FCI) and Conceptual Survey on Electricity and Magnetism that contain distractors that are predicted to be influenced by testwiseness. This analysis was performed using data sets collected between fall 2001 and spring 2014 at one midwestern U.S. university (including over 9500 students) and between Spring 2011 and Spring 2015 at a second eastern U.S. university (including over 2500 students). Student avoidance of “none of the above” or “zero” distractors was statistically significant. The effect of the position of a distractor on its likelihood to be selected was also significant. The effects of several potential positive and negative testwiseness effects on student scores were also examined by developing two modified versions of the FCI designed to include additional elements related to testwiseness; testwiseness produced little effect post-instruction in student performance on the modified instruments

Behavioral Self-Regulation in a Physics Class

This study examined the regulation of out-of-class time invested in the academic activities associated with a physics class for 20 consecutive semesters. The academic activities of 1676 students were included in the study. Students reported investing a semester average of 6.5 2.9 h out of class per week. During weeks not containing an examination, a total of 4.3 2.1 h was reported which was divided between 2.5. 1.2 h working homework and 1.8 1.4 h reading. Students reported spending 7.6. 4.8 h preparing for each in-semester examination. Students showed a significant correlation between the change in time invested in examination preparation (r ¼ −0.12, p \u3c 0.0001) and their score on the previous examination. The correlation increased as the data were averaged over semester (r ¼ −0.70, p ¼ 0.0006) and academic year (r ¼ −0.82, p ¼ 0.0039). While significant, the overall correlation indicates a small effect size and implies that an increase of 1 standard deviation of test score (18%) was related to a decrease of 0.12 standard deviations or 0.9 h of study time. Students also modified their time invested in reading as the length of the textbook changed; however, this modification was not proportional to the size of the change in textbook length. Very little regulation of the time invested in homework was detected either in response to test grades or in response to changes in the length of homework assignments. Patterns of regulation were different for higher performing students than for lower performing students with students receiving a course grade of “C” or “D” demonstrating little change in examination preparation time in response to lower examination grades. This study suggests that homework preparation time is a fixed variable while examination preparation time and reading time are weakly mutable variables

Multi-Dimensional Item Response Theory and the Force Concept Inventory

Research on the test structure of the Force Concept Inventory (FCI) has largely been performed with exploratory methods such as factor analysis and cluster analysis. Multi-Dimensional Item Response Theory (MIRT) provides an alternative to traditional Exploratory Factor Analysis which allows statistical testing to identify the optimal number of factors. Application of MIRT to a sample of $N=4,716$ FCI post-tests identified a 9-factor solution as optimal. Additional analysis showed that a substantial part of the identified factor structure resulted from the practice of using problem blocks and from pairs of similar questions. Applying MIRT to a reduced set of FCI items removing blocked items and repeated items produced a 6-factor solution; however, the factors had little relation the general structure of Newtonian mechanics. A theoretical model of the FCI was constructed from expert solutions and fit to the FCI by constraining the MIRT parameter matrix to the theoretical model. Variations on the theoretical model were then explored to identify an optimal model. The optimal model supported the differentiation of Newton's 1st and 2nd law; of one-dimensional and three-dimensional kinematics; and of the principle of the addition of forces from Newton's 2nd law. The model suggested by the authors of the FCI was also fit; the optimal MIRT model was statistically superior

Exploring the Structure of Misconceptions in the Force Concept Inventory with Modified Module Analysis

Module analysis for multiple-choice responses (MAMCR) was applied to a large sample of Force Concept Inventory (FCI) pretest and post-test responses (Npre ¼ 4509 and Npost ¼ 4716) to replicate the results of the original MAMCR study and to understand the origins of the gender differences reported in a previous study of this dataset. When the results of MAMCR could not be replicated, a modification of the method was introduced, modified module analysis (MMA). MMA was productive in understanding the structure of the incorrect answers in the FCI, identifying 9 groups of incorrect answers on the pretest and 11 groups on the post-test. These groups, in most cases, could be mapped on to common misconceptions used by the authors of the FCI to create distractors for the instrument. Of these incorrect answer groups, 6 of the pretest groups and 8 of the post-test groups were the same for men and women. Two of the male-only pretest groups disappeared with instruction while the third male-only pretest group was identified for both men and women postinstruction. Three of the groups identified for both men and women on the post-test were not present for either on the pretest. The rest of the identified incorrect answer groups did not represent misconceptions, but were rather related to the blocked structure of some FCI items where multiple items are related to a common stem. The groups identified had little relation to the gender unfair items previously identified for this dataset, and therefore, differences in the structure of student misconceptions between men and women cannot explain the gender differences reported for the FCI

The impact of targeting TRAF2 and NCK-interacting protein kinase (TNIK) on anti-tumor effect in small cell lung cancer

View full abstracthttps://openworks.mdanderson.org/leading-edge/1056/thumbnail.jp

Pair-breaking quantum phase transition in superconducting nanowires

A quantum phase transition (QPT) between distinct ground states of matter is a wide-spread phenomenon in nature, yet there are only a few experimentally accessible systems where the microscopic mechanism of the transition can be tested and understood. These cases are unique and form the experimentally established foundation for our understanding of quantum critical phenomena. Here we report the discovery that a magnetic-field-driven QPT in superconducting nanowires - a prototypical 1d-system - can be fully explained by the critical theory of pair-breaking transitions characterized by a correlation length exponent $\nu \approx 1$ and dynamic critical exponent $z \approx 2$. We find that in the quantum critical regime, the electrical conductivity is in agreement with a theoretically predicted scaling function and, moreover, that the theory quantitatively describes the dependence of conductivity on the critical temperature, field magnitude and orientation, nanowire cross sectional area, and microscopic parameters of the nanowire material. At the critical field, the conductivity follows a $T^{(d-2)/z}$ dependence predicted by phenomenological scaling theories and more recently obtained within a holographic framework. Our work uncovers the microscopic processes governing the transition: The pair-breaking effect of the magnetic field on interacting Cooper pairs overdamped by their coupling to electronic degrees of freedom. It also reveals the universal character of continuous quantum phase transitions.Comment: 22 pages, 5 figure